Nonparametric Composite Hypothesis Testing in an Asymptotic Regime

We investigate the nonparametric, composite hypothesis testing problem for arbitrary unknown distributions and in the asymptotic regime with the number of hypotheses grows exponentially large. Such type of asymptotic analysis is important in many practical problems, where the number of variations that can exist within a family of distributions can be countably infinite. We introduce the notion of discrimination capacity, which captures the largest exponential growth rate of the number of hypotheses (with the number of samples) so that there exists a test with asymptotically zero probability of error. Our approach is based on various distributional distance metrics in order to incorporate the generative model of the data. We provide example analyses of the error exponent using the maximum mean discrepancy (MMD) and Kolmogorov-Smirnov (KS) distances and characterized the corresponding discrimination rates (i.e., lower bounds on the discrimination capacity) for these tests. Finally, We provide an upper bound on the discrimination capacity based on Fano’s inequality. Index Terms Nonparametric hypothesis testing, channel coding, maximum mean discrepancy, Kolmogorov-Smirnov distance, error exponent This material is based upon work supported in part by the Defense Advanced Research Projects Agency under Contract No. HR0011-16-C-0135. The work of Y. Liang was also supported in part by the National Science Foundation under grant CCF-1801855. Q. Li, T. Wang, B. Chen, and P. K. Varshney are with the Department of Electrical Engineering and Computer Science, Syracuse University, Syracuse, NY 13244, USA (e-mail: [email protected]; [email protected]; [email protected]; [email protected]). D. J. Bucci is with Lockheed Martin Advanced Technology Labs, Cherry Hill, NJ 08002, USA (e-mail: [email protected]). Y. Liang is with the Department of Electrical and Computer Engineering, The Ohio State University, Columbus, OH 43210, USA (e-mail: [email protected]). ar X iv :1 71 2. 03 29 6v 1 [ cs .I T ] 8 D ec 2 01 7